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NATION 



(57) Abstract 

A system and method for generating a three-dimensional visualization image 
of an object such as an organ using volume visualization techniques and exploring 
the image using a guided navigation system which allows the operator to travel 
along a flight path and to adjust the view to a particular portion of the image of 
interest in order, for example, to idenUfy polyps, cysts or other abnormal features in 
the visualized organ. An electronic biopsy can also be performed on an identified 
growth or mass in the visualized object Virtual colonoscopy can be enhanced by 
electronically removing residual stool, fluid and non-colonic tissue from the image 
of the colon, by employing bowel preparation followed by image segmentation 
operations. Methods are also employed for virtually expanding regions of colon 
collapse using image segmentation results. 



( START J 



REGIONAL PREPARATION 
FOR IMAGING 



I 



^510 



ACQUIRE IMAGE DATA 



1520 



CONVERT IMAGE 
DATA TO VOXELS 



^530 



GROUP VOXELS BY 
INTENSITY THRESHOLD AS 
VOXEL CLUSTERS 



^540 



IDENTIFY TARGET CLUSTER 
FOR REGION OF INTEREST 



PERFORM FEATURS 
VECTOR ANALYSIS ON 
TARGETED CLUSTERIS) 



CLASSIFY VOXELS 
WITHIN CLUSTER 



M5SO 



^15TO 



PERFORM HIGH LEVEL 
FEATURE EXTRACTION 



^1680 



BEST AVAILABLE COPY 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify Sates party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Skrveoia 


AM 


Armenia 


n 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


AU 


Austnlii 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


Untied Kingdom 


MC 


Monaco 


TO 


Chad 


BA 


Bern 11 tnd Herzegovina 


GE 


GttJTJU 


MD 


Republic of Moldova 


TC 


Togo 


BB 


Barbados 


GH 


Ghm 


MC 


Madagascar 


TJ 


Tajikistan 


BE 


Bdgiam 


CN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Fuo 


CR 


Greece 




Republic of Macedonia 


TR 


Turkey 


EC 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


re 


bclmd 


MN 


Mongolia 


UA 


Ukraine 


BR 


Bnzil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Be Una 


IS 


Iceland 


MW 


Malawi 


US 


United Slates of America 


CA 


Canada 


rr 


Italy 


MX 


Mateo 


UZ 


Uzbekistan 


cr 


Central African Republic 


jp 


Japan 


NE 


Niger 


VN 


Viet Nan 


cc 


Congo 


K£ 


Kenya 


NL 


Netherlands 


YU 


Yugoslavii 


CH 


Switzerland 


KG 


Kyrgyzsran 


NO 


Norway 


ZW 


Zimbabwe 


a 


Core d'lvoirt 


KP 


Democmic People "» 


NZ 


New Zealand 






CM 


Curt croon 




Republic of Korea 


PL 


Poland 






CN 


Gnu 


KR 


Repobbc of Korea 


FT 


Portugal 






CD 


Ccba 


KZ 


Kazakstan 


RO 


Romania 






a 


Czech Republic 


LC 


Sum Locu 


RU 


Russian Federation 






DE 


Gernuny 


U 


Lccrcenstea 


SO 


Sudan 






OK 




LK 


SriLaaka 


SE 








EE 


Esutnii 


LR 


Ubena 


SC 


Singapore 







WO 00/55814 PCT/USOO/07352 

1 

SYSTEM AND METHOD FOR PERFORMING A THREE-DIMENSIONAL 
VIRTUAL SEGMENTATION AND EXAMINATION 

SPECIFICATION 



ICAL FIELD 

5 The present invention relates to a system and method for performing a 

volume based three-dimensional virtual examination using planned and guided 
navigation techniques. One such application is performing a virtual endoscopy. 

BACKGROUND O F THE INVENTION 
Colon cancer continues to be a major cause of death throughout the 

10 world. Early detection of cancerous growths, which in the human colon initially 
manifest themselves as polyps, can greatly improve a patient's chance of recovery. 
Presently, there are two conventional ways of detecting polyps or other masses in the 
colon of a patient. The first method is a colonoscopy procedure, which uses a flexible 
fiber-optic tube called a colonoscope to visually examine the colon by way of physical 

15 rectal entry with the scope. The doctor can manipulate the tube to search for any 
abnormal growths in the colon. The colonoscopy, although reliable, is both relatively 
costly in money and ume, and is an invasive, uncomfortable painful procedure for the 
patient 

The second detection technique is the use of a barium enema and two- 
20 dimensional X-ray imaging of the colon. The barium enema is used to coat the colon 
with barium, and a two-dimensional X-ray image is taken to capture an image of the 
colon. However, barium enemas may not always provide a view of the entire colon, 
require extensive pretreatment and patient manipulation, is often operator-dependent 
when performing the operation, exposes the patient to excessive radiation and can be 
25 less sensitive than a colonoscopy. Due to deficiencies in the conventional practices 
described above, a more reliable., less intrusive and less expensive way to check the 
colon for polyps is desirable. A method to examine other human organs, such as the 



WO 00/55814 PCT/USOO/07352 

2 

lungs, for masses in a reliable, cost effective way and with less patient discomfort is 
also desirable. 

Two-dimensional ("2D") visualization of human organs employing 
currently available medical imaging devices, such as computed tomography and MRI 

S (magnetic resonance imaging), has been widely used for patient diagnosis. Three- 
dimensional images can be formed by stacking and interpolating between two- 
dimensional pictures produced from the scanning machines. Imaging an organ and 
visualizing its volume in three-dimensional space would be beneficial due to its lack 
of physical intrusion and the ease of data manipulation. However, the exploration of 

1 0 the three-dimensional volume image must be properly performed in order to fully 
exploit the advantages of virtually viewing an organ from the inside. 

When viewing the three dimensional ("3D") volume virtual image of 
an environment, a functional model must be used to explore the virtual space. One 
possible model is a virtual camera which can be used as a point of reference for the 

15 viewer to explore the virtual space. Camera control in the context of navigation 
within a general 3D virtual environment has been previously studied. There are two 
conventional types of camera control offered for navigation of virtual space. The first 
gives the operator full control of the camera which allows the operator to manipulate 
the camera in different positions and orientations to achieve the view desired. The 

20 operator will in effect pilot the camera. This allows the operator to explore a 
particular section of interest while ignoring other sections. However, complete 
control of a camera in a large domain would be tedious and tiring, and an operator 
might not view all the important features between the start and finishing point of the 
exploration. The camera could also easily get "lost" in remote areas or be "crashed" 

25 into one of the walls by an inattentive operator or by numerous unexpected obstacles. 

The second technique of camera control is a planned navigation 
method, which assigns the camera a predetermined path to take and which cannot be 
changed by the operator. This is akin to having an engaged "autopilot". This allows 
the operator to concentrate on the virtual space being viewed, and not have to worry 

30 about steering into walls of the environment being examined. However, this second 
technique does not give the viewer the flexibility to alter the course or investigate an 
interesting area viewed along the flight path. 
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It would be desirable to use a combination of the two navigation 
techniques described above to realize the advantages of both techniques while 
minimizing their respective drawbacks. It would be desirable to apply a flexible 
navigation technique to the examination of human or animal organs which are 
5 represented in virtual 3D space in order to perfoim a non-intrusive painless thorough 
examination. The desired navigation technique would further allow for a complete 
examination of a virtual organ in 3D space by an operator allowing flexibility while 
ensuring a smooth path and complete examination through and around the organ. It 
would be additionally desirable to be able to display the exploration of the organ in a 
10 real time setting by using a technique which minimizes the computations necessary for 
viewing the organ. The desired technique should also be equally applicable to 
exploring any virtual object. 

SUMMARY 

The invention generates a three-dimensional visualization image of an 
1 5 object such as a human organ using volume visualization techniques and explores the 
virtual image using a guided navigation system which allows the operator to travel 
along a predefined flight path and to adjust both the position and viewing angle to a 
particular portion of interest in the image away from the predefined path in order to ' 
identify polyps, cysts or other abnormal features in the organ. 
20 ^ inventive technique for three-dimensional virtual examination of 

an object includes producing a discrete representation of the object in volume 
elements, defining the ponion of the object which is to be examined, performing a 
navigation operation in the virtual object and displaying the virtual object in real time 
during the navigation. 

25 inventive technique for a three-dimensional virtual examination as 

applied to an organ of a patient includes preparing the organ for scanning, if 
necessary, scanning the organ and converting the data into volume elements, defining 
the portion of the organ which is to be examined, performing a guided navigation 
operation in the virtual organ and displaying the virtual organ in real time during the 

30 guided navigation. 
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In performing virtual examination, it is often desirable to view a 
particular material type while removing other material types from the image. To 
perform such an operation, a method for electronically cleansing an image can be 
performed by converting the image data to a plurality of volume elements with each 
5 volume element having an intensity value. Next , a classifying operation is performed 
to classify the volume elements into a plurality of clusters in accordance with the 
intensity values. Once classified, at least one cluster of volume elements can then be 
removed from the image data. 

The classifying operation can be performed by evaluating a plurality of 
10 volume elements of the image data with respect to a plurality of neighboring volume 
elements to determine a neighborhood similarity value for the volume element 

The clusters can be further classified by applying a mixture probability 
function to the clusters to classify voxels whose intensity value results from inclusion 
of more than one material type. 
1 5 An alternative classifying operation includes the steps of performing 

feature vector analysis on at least one of the clusters which include image data for a 
material of interest followed by performing high level feature extraction to remove 
volume elements from the image which are not substantially indicative of the material 
of interest 

20 The method according method for electronically cleansing an image is 

well suited for applications where the image data represents a region of the human 
body including at least a portion of the colon and the material of interest is tissue of a 
colon. In colon imaging applications, the removing operation can remove volume 
elements representing intracolonic fluid, residual stool within the colon, bone, and 

25 non-colonic tissue. 

It is an object of the invention to use a system and method to perform a 
relatively painless, inexpensive and non-intrusive in vivo examination of an organ 
where the actual analysis of the scanned colon can be possibly performed without the 
patient present. The colon can be scanned and visualized in real-time or the stored 

30 data can be visualized at a later time. 
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It is another object of the invention to generate 3D volume 
representations of an object, such as an organ, where regions of the object can be 
peeled back layer by layer in order to provide sub-surface analysis of a region of the 
imaged object. A surface of an object (such as an organ) can be rendered transparent 
5 or translucent in order to view further objects within or behind the object wall. The 
object can also be sliced in order to examine a particular cross-section of the object. 

It is another object of the invention to provide a system and method of 
guided navigation through a 3D volume representation of an object, such as an organ, 
using potential fields. 

It is a further object of the invention to calculate the center-line of an 
object, such as an organ, for a virtual fly-through using a peel-layer technique as 
described herein. 

It is still a further object of the invention to use a modified Z buffer 
technique to minimize the number of computations required for generating the 
15 viewing screen. 

It is another object of the invention to assign opacity coefficients to 
each volume element in the representation in order to make particular volume 
elements transparent or translucent to varying degrees in order to customize the 
visualization of the portion of the object being viewed. A section of the object can 
20 also be composited using the opacity coefficients. 



10 



BRIEF DESCRIPTION nr t » E DRAWING 
Further objects, features and advantages of the invention will become 
apparent from the following detailed description taken in conjunction with the 
accompanying figures showing a preferred embodiment of the invention, on which: 
25 F '8 ure 1 is a How chart of the steps for performing a virtual 

exajnination of an object, specifically a colon, in accordance with the invention; 

Figure 2 is an illustration of a "submarine" camera model which 
performs guided navigation in the virtual organ; 

Figure 3 is an illustration of a pendulum used to model pitch and roll of 
30 the "submarine" camera; 
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Figure 4 is a diagram illustrating a two dimensional cross-section of a 
volumetric colon which identifies two blocking walls; 

Figure 5 is a diagram illustrating a two dimensional cross-section of a 
volumetric colon upon which start and finish volume elements are selected; 
5 Figure 6 is a diagram illustrating a two dimensional cross-section of a 

volumetric colon which shows a discrete sub-volume enclosed by the blocking walls 
and the colon surface; 

Figure 7 is a diagram illustrating a two dimensional cross-section of a 
volumetric colon which has multiple layers peeled away; 
1 0 Figure 8 is a diagram illustrating a two dimensional cross-section of a 

volumetric colon which contains the remaining flight path; 

Figure 9 is a flow chart of the steps of generating a volume 
visualization of the scanned organ; 

Figure 10 is an illustration of a virtual colon which has been sub- 
15 divided into cells; 

Figure 1 1 A is a graphical depiction of an organ which is being virtually 

examined; 

Figure 1 IB is a graphical depiction of a sub tree generated when 
depicting the organ in Fig. 1 1 A; 
20 Figure 1 1 C is a further graphical depiction of a stab tree generated 

while depicting the organ in Fig. 1 1 A; 

Figure 12A is a graphical depiction of a scene to be rendered with 
objects within certain cells of the scene; 

Figure 12B is a graphical depiction of a stab tree generated while 
25 depicting the scene in Fig. 12A; 

Figures 12C-12E are further graphical depictions of stab trees 
generated while depicting the image in Fig. 12 A; 

Figure 1 3 is a two dimensional representation of a virtual colon 
containing a polyp whose layen can be removed; 
30 Figure 14 is a diagram of a system used to perform a virtual 

examination of a human organ in accordance with the invention; 
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Figure 15 is a flow chart depicting an improved image segmentation 

method; 

Figure 16 is a graph of voxel intensity versus frequency of a typical 
abdominal CT data set; 

5 Figure 1 7 is a perspective view diagram of an intensity vector structure 

including a voxel of interest and its selected neighbors; 

Figure 1 8A is an exemplary image slice from a CT scan of a human 
abdominal region, primarily illustrating a region including the lungs; 

Figure 18B is a pictorial diagram illustrating the identification of the 
1 0 lung region in the image slice of Figure J 8 A; 

Figure 18C is a pictorial diagram illustrating the removal of the lung 
volume identified in Figure 18B; 

Figure 19A is a exemplary image slice form a CT scan of a human 
abdominal region, primarily illustrating a region including a portion of the colon and 
15 bone; 

Figure 19B is a pictorial diagram illustrating the identification of the 
colon and bone region from the image slice of Figure 19 A; 

Figure 19C is a pictorial diagram illustrating the image scan of figure 
19a with the regions of bone removed; and 

Figure 20 is a flowchart illustrating a method for applying texture to 
monochrome image data. 



20 



DETAILED nESCRlPTinx] 
While the methods and systems described in this application can be 
applied to any object to be examined, the preferred embodiment which will be 
25 described is the examination of an organ in the human body, specifically the colon. 
The colon is long and twisted which makes it especially suited for a virtual 
examination saving the patient both money and the discomfort and danger of a 
physical probe. Other examples of organs which can be examined include the lungs, 
stomach and portions of the gastro-intestinal system, the heart and blood vessels. 
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Fig. 1 illustrates the steps necessary to perform a virtual colonoscopy 
using volume visualization techniques. Step 101 prepares the colon to be scanned in 
order to be viewed for examination if required by either the doctor or the particular 
scanning instrument. This preparation could include cleansing the colon with a 

5 "cocktail" or liquid which enters the colon after being orally ingested and passed 
through the stomach. The cocktail forces the patient to expel waste material that is 
present in the colon. One example of a substance used is Golytely. Additionally, in 
the case of the colon, air or CO, can be forced into the colon in order to expand it to 
make the colon easier to scan and examine. This is accomplished with a small tube 

1 0 placed in the rectum with approximately 1 ,000 cc of air pumped into the colon to 
distend the colon. Depending upon the type of scanner used, it may be necessary for 
the patient to drink a contrast substance such as barium to coat any unexpunged stool 
in order to distinguish the waste in the colon from the colon walls themselves. 
Alternatively, the method for virtually examining the colon can remove the virtual 

1 5 waste prior to or during the virtual examination as explained later in this specification. 
Step 101 does not need to be performed in all examinations as indicated by the dashed 
line in Fig. 1. 

Step 1 03 scans the organ which is to be examined. The scanner can be 
an apparatus well known in the art such as a spiral CT-scanner for scanning a colon 

20 or a Zenita MR1 machine for scanning a lung labeled for example with xenon gas. 
The scanner must be able to take multiple images from different positions around the 
body during suspended respiration, in order to produce the data necessary for the 
volume visualization. An example of a single CT-image would use an X-ray beam of 
5mm width, 1:1 to 2: 1 pitch, with a 40cm field-of-view being performed from the top 

25 of the splenic flexure of the colon to the rectum. 

Discrete data representations of said object can be produced by other 
methods besides scanning. Voxel data representing an object can be derived from a 
geometric model by techniques described in U.S. Pat No. 5,038,302 entitled "Method 
of Converting Continuous Three-Dimensional Geometrical Representations into 

30 Discrete Three-Dimensional Voxel-Based Representations Within a Three- 
Dimensional Voxel-Based System" by Kaufman, issued Aug. 8, 1991, filed July 26, 
1988, which is hereby incorporated by reference. Additionally, data can be produced 
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by a computer model of an image which can be converted to three-dimension voxels 
and explored in accordance with this invention. One example of this type of data is a 
computer simulation of the turbulence surrounding a space shuttle craft. 

Step 104 converts the scanned images into three-dimensional volume 
elements (Voxels). In the preferred embodiment for examining a colon, the scan data 
is refomiatted into 5mm thick slices at increments of 1mm or 2.5mm, with each slice 
represented as a matrix of 512 by 512 pixels. Thus a large number of 2D slices are 
generated depending upon the length of the scan. The set of 2D slices is then 
reconstructed to 3D voxels. The conversion process of 2D images from the scanner 
into 3D voxels can either be performed by the scanning machine itself or by a separate 
machine such as a computer with techniques which are well known in the art (for 
example, see U.S. Pat. No. 4,985,856 entitled "Method and Apparatus for Storing, 
Accessing, and Processing Voxel-based Data" by Kaufman et al.; issued Jan. 15, 
1991 , filed Nov. 1 1 , 1988; which is hereby incorporated by reference). 

Step 105 allows the operator to define the portion of the selected organ ' 
to be examined. A physician may be interested in a particular section of the colon 
likely to develop polyps. The physician can view a two dimensional slice overview 
map to indicate the section to be examined. A starting point and finishing point of a 
path to be viewed can be indicated by the physician/operator. A conventional 
computer and computer interface (e.g., keyboard, mouse or spaceball) can be used to 
designate the portion of the colon which is to be inspected. A grid system with 
coordinates can be used for keyboard entry or the physician/operator can "click" on 
the desired points. The entire image of the colon can also be viewed if desired. 

Step 1 07 performs the planned or guided navigation operation of the 
virtual organ being examined. Performing a guided navigation operation is defined as 
navigating through an environment along a predefined or automatically predetermined 
flight path which can be manually adjusted by an operator at any time. After the scan 
data has been converted to 3D voxels, the inside of the organ must be traversed from 
the selected start to the selected finishing point. The virtual examinations is modeled 
on having a tiny camera traveling through the virtual space with a lens pointing 
towards the finishing point. The guided navigation technique provides a level of 
interaction with the camera, so that the camera can navigate through a virtual 
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environment automatically in the case of no operator interaction, and at the same time, 
allow the operator to manipulate the camera when necessary. The preferred 
embodiment of achieving guided navigation is to use a physically based camera model 
which employs potential fields to control the movement of the camera and which are 
5 described in detail in Figs. 2 and 3. 

Step 1 09, which can be performed concurrently with step 107, displays 
the inside of the organ from the viewpoint of the camera model along the selected 
pathway of the guided navigation operation. Three-dimensional displays can be 
generated using techniques well known in the art such as the marching cubes 
1 0 technique. However, in order to produce a real time display of the colon, a technique 
is required which reduces the vast number of computations of data necessary for the 
display of the virtual organ. Fig. 9 describe this display step in more detail. 

The method described in Figure 1 can also be applied to scanning 
multiple organs in a body at the same time. For example, a patient may be examined 
15 for cancerous growths in both the colon and lungs. The method of Figure 1 would be 
modified to scan all the areas of interest in step 103 and to select the current organ to 
be examined in step 1 05. For example, the physician/operator may initially select the 
colon to virtually explore and later explore the lung. Alternatively, two different 
doctors with different specialties may virtually explore different scanned organs 
20 relating to their respective specialties. Following step 1 09, the next organ to be 
examined is selected and its portion will be defined and explored. This continues 
until all organs which need examination have been processed. 

The steps described in conjunction with Figure 1 can also be applied to 
the exploration of any object which can be represented by volume elements. For 
25 example, an architectural structure or inanimate object can be represented and 
explored in the same manner. 

Figure 2 depicts a "submarine" camera control model which performs 
the guided navigation technique in step 107. When there is no operator control during 
guided navigation, the default navigation is similar to that of planned navigation 
30 which automatically directs the camera along a flight path from one selected end of 
the colon to another. During the planned navigation phase, the camera stays at the 
center of the colon for obtaining better views of the colonic surface. When an 
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interesting region is encountered, the operator of the virtual camera using guided 
navigation can interactively bring the camera close to a specific region and direct the 
motion and angle of the camera to study the interesting area in detail, without 
unwillingly colliding with the walls of the colon. The operator can control the camera 
5 with a standard interface device such as a keyboard, mouse or non-standard device 
such as a spaceball. In order to fully operate a camera in a virtual environment, six 
degTees of freedom for the camera is required. The camera must be able to move in 
the horizontal, vertical, and Z direction (axes 217), as well as being able to rotate in 
another three degrees of freedom (axes 219) to allow the camera to move and scan all 
10 sides and angles of a virtual environment The camera model for guided navigation 
includes an inextensible, weightless rod 201 connecting two particles x, 203 and x 2 
205, both particles being subjected to a potential field 215. The potential field is 
defined to be highest at the walls of the organ in order to push the camera away from 
the walls. 

1 5 1116 Positions of the particles are given by x, and x„ and they are 

assumed to have the same mass m. A camera is attached at the head of the submarine 
x, 203, whose viewing direction coincides with XjX,. The submarine can perform 
translation and rotation around the center of mass x of the model as the two particles 
are affected by the forces from the potential field V(x) which is defined below, any 

20 friction forces, and any simulated external force. The relations between x„ x>, and x 
are as follows: 

x = (xyj), 

r - (rsin6cos<p,ninBsin<f),rcos6), 
x, = x + r, 

25 x, = x-r, (i) 



where r, 6 and <}> are the polar coordinates of the vector xx,. 

The kinetic energy of the model, T, is defined as the summation of the kinetic energies 

of the movements of x, and x } : 



I 
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= mi 5 -f- mr* 

= m(i , + y , + i , )+mr , (P , + ^sin , «). (2) 
Then, the equations for the motion of the submarine model are obtained by using 

LaGrange's equation: 

where the q^s are the generalized coordinates of the model and can be considered as 
the variables of time / as: 



5 with i|r denoting the roll angle of our camera system, which will be explained later. 
The FjS are called the generalized forces. The control of the submarine is performed 
by applying a simulated external force to x„ 

and it is assumed that both x, and x ; are affected by the forces from the potential field 
10 and the frictions which act in the opposite direction of each panicle's velocity. Conse- 
quently, the generalized forces are formulated as follows: 

*CXtl 

Fj = -mWM-kxj, (5) 



where k denotes the friction coefficient of the system. The external force F 01 is 
applied by the operator by simply clicking the mouse button in the desired direction 
207 in the generated image, as shown in Figure 2. This camera model would then be 
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moved in that direction. This allows the operator to control at least five degrees of 
freedom of the camera with only a single click of the mouse button. From Equations 
(2), (3) and (5), it can be derived that the accelerations of the five parameters of our 
submarine model as: 



z = 



_i f av(x,) gvfo) n r. 

2 V fly + ^T ) *m + 2^' 
-If fl^xi) , gVfa^, fci F, 

./ flx + ~aT- ) -m + ^ 

*a 1 , 

m + 008 6 008 * + F " 9 F ' "» A). 
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fl» 
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(6) 



5 where i and x denote the first and the second derivative of x, respectively, and 

( dvix) ey(x) dV(x)\ . . 
[ 5 X ' g y » g z I den °tM the gradient of the potential i 



! at a point x. 



The terms $ 2 sin8cos6 of 6 and - 29< "' cose of 5 are called the 

sind 
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centrifugal force and the Coriolis force, respectively, and they are concerned with the 
exchange of angular velocities of the submarine. Since the model does not have the 
moment of inertia defined for the rod of the submarine, these terms tend to cause an 
overflow of the numeric calculation of <|>. Fortunately, these terms become significant 

5 only when the angular velocities of the submarine model are significant, which 

essentially means that the camera moves too fast Since it is meaningless to allow the 
camera to move so fast because the organ could not be properly viewed, these terms 
are minimized in our implementation to avoid the overflow problem. 

From the first three formulas of Equation (6), it is known that the 

1 0 submarine cannot be propelled by the external force against the potential field if the 
following condition is satisfied: 

|V7(xO + W(x 3 )|>^i. 



Since the velocity of the submarine and the external force have upper limits in our 
implementation, by assigning sufficiently high potential values at the boundary of the 
objects, it can be guaranteed that the submarine never bumps against the objects or 

1 5 walls in the environment. 

As mentioned previously, the roll angle of the camera system needs 
to be considered. One possible option allows the operator full control of the angle \Jj. 
However, although the operator can rotate the camera freely around the rod of the 
model, he or she can easily become disoriented. The preferred technique assumes that 

20 the upper direction of the camera is connected to a pendulum with mass m 2 30 1 , which 
routes freely around the rod of the submarine, as shown in Figure 3. The direction of 
the pendulum, r 2 , is expressed as: 



w 
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r a = r 2 (cos0cos^sb^ + »n^ccs^ f cos0sia^siiiV>- cos^cos^,- sinflsin ). 



although it is possible to calculate the accurate movement of this pendulum along with 
the movement of the submarine, it makes the system equations too complicated. 
Therefore, it is assumed that all the generalized coordinates except the roll angle \|r are 
constants, and thus define the independent kinetic energy for the pendulum system as: 

5 This simplifies the model for the roll angle. Since it is assumed in this model that the 
gravitational force 

F, = m 2 g = (m 2 0 r ,77l2$ y , m 2 02 ) 

acts at the mass point m 2 , the acceleration of ip can be derived using LaGrange's 
equation as: 

1 

$ = ^{<fe(cos0cospcosV> - sin^sin^) 
+<h (cos 0 sin <f> cos ^ + cos ^ sin ^) 

+S,(-sto0COS^}- (7) 

From Equations (6) and (7), the generalized 



coordinates q(t) and their derivatives q(t) are calculated asymptotically by 
Taylor series as: 
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q(t+i) = q(<)+%)+yq(0+0(k 8 ), 

q(f + fc) = q(f) + fcq(i) + 0(Ji s ), 

to freely move the submarine. To smooth the submarine's motion, the time step h is 
selected as an equilibrium value between being as small as possible to smooth the 
motion but as large as necessary to reduce computation cost 

Definition of the Potential Field 
5 The potential field in the submarine model in Figure 2 defines the 

boundaries (walls or other matter) in the virtual organ by assigning a high potential to 
the boundary in order to ensure that the submarine camera does not collide with the 
walls or other boundary. If the camera model is attempted to be moved into a high 
potential area by the operator, the camera model will be restrained from doing so 

10 unless the operator wishes to examine the organ behind the boundary or inside a 

polyp, for example. In the case of performing a virtual colonoscopy, a potential field 
value is assigned to each piece of volumetric colon data (volume element). When a 
particular region of interest is designated in step 105 of Fig. 1 with a start and finish 
point, the voxels within the selected area of the scanned colon are identified using 

1 5 conventional blocking operations. Subsequently, a potential value is assigned to every 
voxel x of the selected volume based on the following three distance values: the 
distance from the finishing point dt(x), the distance from the colon surface ds(x) and 
the distance from the center-line of the colon space dc(x). dt(x) is calculated by using 
a conventional growing strategy. The distance from the colon surface, ds(x), is 

20 computed using a conventional technique of growing from the surface voxels inwards. 
To determine dc(x), the center-line of the colon from the voxel is first extracted, and 
then dc(x) is computed using the conventional growing strategy from the center-line 
of the colon. 

To calculate the center-line of the selected colon area defined by the 
25 user-specified start point and the user-specified finish point, the maximum value of 
ds(x) is located and denoted dmax. Then for each voxel inside the area of interest, a 
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cost value of dmax - ds(x) is assigned. Thus the voxels which are close to the colon 
surface have high cost values and the voxels close to the center line have relatively 
low cost values. Then, based on the cost assignment, the single-source shortest path 
technique which is well known in the an is applied to efficiently compute a minimum 
5 cost path from the source point to the finish point. This low cost line indicates the 
center-line or skeleton of the colon section which is desired to be explored. This 
technique for determining the center-line is the preferred technique of the invention. 

To compute the potential value V(x) for a voxel x inside the area of 
interest, the following formula is employed: 



10 



15 



20 



V(x) = di&r + g»( T7T~n J ~"> 



where C„ C» ji and v are constants chosen for the task. In order to avoid any 
collision between the virtual camera and the virtual colonic surface, a sufficiently ' 
large potential value is assigned for all points outside the colon. The gradient of the 
potential field will therefore become so significant that the submarine model camera 
will never collide with the colonic wall when being run. 

Another technique to determine the center-line of the path in the colon 
is called the "peel-layer" technique and is shown in Figure 4 through Figure 8. 

Figure 4 shows a 2D cross-section of the volumetric colon, with the 
two side walls 401 and 403 of the colon being shown. Two blocking walls are 
selected by the operator in order to define the section of the colon which is of interest 
to examine. Nothing can be viewed beyond the blocking walls. This helps reduce the 
number of computations when displaying the virtual representation. The blocking 
walls together with side walls identify a contained volumetric shape of the colon 
which is to be explored. 

Figure 5 shows two end points of the flight path of the virtual 
25 examination, the start volume element 501 and the finish volume element 503. The 
start and finish points are selected by the operator in step 105 of Fig. 1. The voxels 
between the start and finish points and the colon sides are identified and marked, as 
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indicated by the area designated with Vs in Fig. 6. The voxels are three-dimensional 
representations of the picture element. 

The peel-layer technique is then applied to the identified and marked 
voxels in Fig. 6. The outermost layer of all the voxels (closest to the colon walls) is 
5 peeled off step-by-step, until there is only one inner layer of voxels remaining. Stated 
differently, each voxel furthest away from a center point is removed if the removal 
does not lead to a disconnection of the path between the start voxel and the finish 
voxel. Figure 7 shows the intermediate result after a number of iterations of peeling 
the voxels in the virtual colon are complete. The voxels closest to the walls of the 
1 0 colon have been removed. Fig. 8 shows the final flight path for the camera model 
down the center of the colon after all the peeling iterations are complete. This 
produces essentially a skeleton at the center of the colon and becomes the desired 
flight path for the camera model. 

Z- Buffer assisted visibility 
1 5 Figure 9 describes a real time visibility technique to display of virtual 

images seen by the camera model in the virtual three-dimensional volume 
representation of an organ. Figure 9 shows a display technique using a modified Z 
buffer which corresponds to step 109 in Fig. 1 . The number of voxels which could be 
possibly viewed from the camera model is extremely large. Unless the total number 
20 of elements (or polygons) which must be computed and visualized is reduced from an 
entire set of voxels in the scanned environment, the overall number of computations 
will make the visualization display process exceedingly slow for a large internal area. 
However, in the present invention only those images which are visible on the colon 
surface need to be computed for display. The scanned environment can be subdivided 
25 into smaller sections, or cells. The Z buffer technique then renders only a portion of 
the cells which are visible from the camera. The Z buffer technique is also used for 
three-dimensional voxel representations. The use of a modified Z buffer reduces the 
number of visible voxels to be computed and allows for the real time examination of 
the virtual colon by a physician or medical technician. 
30 The area of interest from which the center-line has been calculated in 

step 1 07 is subdivided into cells before the display technique is applied. Cells are 
collective groups of voxels which become a visibility unit The voxels in each cell 
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will be displayed as a group. Each cell contains a number of portals through which 
the other cells can be viewed. The colon is subdivided by beginning at the selected 
start point and moving along the center-line 1001 towards the finish point. The colon 
is then partitioned into cells (for example, cells 1003, 1005 and 1007 in Fig. 10) when 
5 a predefined threshold distance along the center-path is reached. The threshold 
distance is based upon the specifications of the platform upon which the visualization 
technique is performed and its capabilities of storage and processing. The cell size is 
directly related to the number of voxels which can be stored and processed by the 
platform. One example of a threshold distance is 5cm, although the distance can 
greatly vary. Each cell has two cross-sections as portals for viewing outside of the cell 
as shown in Fig. 10. 

Step 901 in Fig. 9 identifies the cell within the selected organ which 
currently contains the camera. The current cell will be displayed as well as all other 
cells which are visible given the orientation of the camera. Step 903 builds a stab tree 

15 (tree diagram) of hierarchical data of potentially visible cells from the camera 

(through denned portals), as will be described in further detail hereinbelow. The stab 
tree contains a node for every cell which may be visible to the camera. Some of the 
cells may be transparent without any blocking bodies present so that more than one . 
cell will be visible in a single direction. Step 905 stores a subset of the voxels from a 

20 cell which include the intersection of adjoining cell edges and stores them at the 
outside edge of the stab tree in order to more efficiently determine which cells are 
visible. 

Step 907 checks if any loop nodes are present in the stab tree. A loop 
node occurs when two or more edges of a single cell both border on the same nearby 
cell. Thismayoccurwhenasinglecellissurroundedbyanothercell. Ifaloopnode 
is identified in the stab tree, the method continues with step 909. If there is no loop 
node, the process goes to step 91 1. 

Step 909 collapses the two cells making up the loop node into one 
large node. The stab tree is then corrected accordingly. This eliminates the problem 
of viewing the same cell twice because of a loop node. The step is performed on all 
identified loop nodes. The process then continues with step 91 1. 
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Step 91 1 then initiates the Z-buffer with the largest Z value. The Z 
value defines the distance away from the camera along the skeleton path. The tree is 
then traversed to first check the intersection values at each node. If a node 
intersection is covered, meaning that the currentjxjrtal sequence is occluded (which is 

5 determined by the Z buffer test), then the traversal of the current branch in the tree is 
stopped. Step 913 traverses each of the branches to check if the nodes are covered 
and displays them if they are not. 

Step 915 then constructs the image to be displayed on the operator's 
screen from the volume elements within the visible cells identified in step 913 using 

1 0 one of a variety of techniques known in the art, such as volume rendering by 
compositing. The only cells shown are those which are identified as potentially 
visible. This technique limits the number of cells which requires calculations in order 
to achieve a real time display and correspondingly increases the speed of the display 
for better performance. This technique is an improvement over prior techniques 

1 5 which calculate all the possible visible data points whether or not they are actually 
viewed. 

Figure 1 1 A is a two dimensional pictorial representation of an organ 
which is being explored by guided navigation and needs to be displayed to an 
operator. Organ 1 101 shows two side walls 1 102 and an object 1 105 in the center of 

20 the pathway. The organ has been divided into four cells A 1 1 5 1 , B 1 153, C 1 155 and 
D 1157. The camera 1 103 is facing towards cell D 1 157 and has a field of vision 
defined by vision vectors 1 107, 1 108 which can identify a cone-shaped field. The 
cells which can be potentially viewed are cells B 1 153, C 1 155 and D 1 157. Cell C 
1 155 is completely surrounded by Cell B and thus constitutes a node loop. 

25 Fig. 1 1 B is a representation of a stab tree built from the cells in Fig. 

1 1 A. Node A 1 1 09 which contains the camera is at the root of the tree. A sight line 
or sight cone, which is a visible path without being blocked, is drawn to node B 1 1 10. 
Node B has direct visible sight lines to both node C 1 1 12 and node D 11 14 and which 
is shown by the connecting arrows. The sight line of node C 1 1 12 in the direction of 

30 the viewing camera combines with node B 1 1 10. Node C 1 1 12 and node B 1 1 10 will 
thus be collapsed into one large node B 1 1 122 as shown in Fig. 1 1C. 
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Fig. 1 1C shows node A 1 109 containing the camera adjacent to node B' 
1 122 (containing both nodes B and node C) and node D 1114. The nodes A, B' and D 
will be displayed at least partially to the operator. 

Figs 12A - 12E illustrate the use of the modified Z buffer with cells 
5 that contain objects which obstruct the views. An object could be some waste 
material in a portion of the virtual colon. Fig. 12A shows a virtual space with 
10 potential cells: A 1251, B 1253, C 1255, D 1257, E 1259, F 1261, G 1263, H 1265, 
1 1267 and J 1269. Some of the cells contain objects. If the camera 1201 is 
positioned in cell 1 1267 and is facing toward cell F 1261 as indicated by the vision 

1 0 vectors 1203, then a stab tree is generated in accordance with the technique illustrated 
by the flow diagram in Fig. 9. Fig. 12B shows the stab tree generated with the 
intersection nodes showing for the virtual representation as shown in Fig. 12A. Fig. 
12B shows cell 1 1 267 as the root node of the tree because it contains the camera - * 
1201. Node 1 121 1 is pointing to node F 1213 (as indicated with an arrow), because 

15 cell F is directly connected to the sight line of the camera. Node F 1213 is pointing to 
both node B 1215 and node E 1219. Node B 1215 is pointing to node A 1217. Node 
C 1202 is completely blocked from the line of sight by camera 1201 so is not included 
in the stab tree. 

Fig. 12C shows the stab tree after node I 121 1 is rendered on the 
20 display for the operator. Node 1 1 2 1 1 is then removed from the stab tree because it 
has already been displayed and node F 1213 becomes the root. Fig. 12D shows that 
node F 1213 is now rendered to join node 1 121 1. The next nodes in the tree 
connected by arrows are then checked to see if they are already covered (already 
processed). In this example, all of the intersected nodes from the camera positioned in 
25 cell 1 1267 has been covered so that node B 515 (and therefore dependent node A) do 
not need to be rendered on the display. 

Fig. 12E shows node E 515 being checked to determine if its 
intersection has been covered. Since it has, the only rendered nodes in this example of 
Figure 12A-12E are nodes I and F while nodes A, B and E are not visible and do not 
30 need to have their cells prepared to be displayed. 
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The modified Z buffer technique described in Figure 9 allows for fewer 
computations and can be applied to an object which has been represented by voxels or 
other data elements, such as polygons. 

Figure 13 shows a two dimensional virtual view of a colon with a large 
5 polyp present along one of its walls. Figure 1 3 shows a selected section of a patient's 
colon which is to be examined further. The view shows two colon walls 1301 and 
1 303 with the growth indicated as 1 305. Layers 1 307, 1 309, and 1 3 1 1 show inner 
layers of the growth. It is desirable for a physician to be able to peel the layers of the 
polyp or tumor away to look inside of the mass for any cancerous or other harmful 
1 0 material. This process would in effect perform a virtual biopsy of the mass without 
actually cutting into the mass. Once the colon is represented virtually by voxels, the 
process of peeling away layers of an object is easily performed in a similar manner as 
described in conjunction with Figs. 4 through 8. The mass can also be sliced so that a 
particular cross-section can be examined. In Fig. 13, a planar cut 1313 can be made 
15 so that a particular portion of the growth can be examined. Additionally, a user- 
defined slice 1 3 1 9 can be made in any manner in the growth. The voxels 1 3 1 9 can 
either be peeled away or modified as explained below. 

A transfer function can be performed to each voxel in the area of 
interest which can make the object transparent, semi-transparent or opaque by altering 
20 coefficients representing the translucently for each voxel. An opacity coefficient is 
assigned to each voxel based on its density. A mapping function then transforms the 
density value to a coefficient representing its translucency. A high density scanned 
voxel will indicate either a wall or other dense matter besides simply open space. An 
operator or program routine could then change the opacity coefficient of a voxel or 
25 group of voxels to make them appear transparent or semi-transparent to the submarine 
camera model. For example, an operator may view a tumor within or outside of an 
entire growth. Or a transparent voxel will be made to appear as if it is not present for 
the display step of Figure 9. A composite of a section of the object can be created 
using a weighted average of the opacity coefficients of the voxels in that section. 
30 If a physician desires to view the various layers of a polyp to look for a 

cancerous areas, this can be performed by removing the outer layer of polyp 1305 
yielding a first layer 1307. Additionally, the first inner layer 1307 can be stripped 
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back to view second inner layer 1309. The second inner layer can be stripped back to 
view third inner layer 13 1 1 , etc. The physician could also slice the polyp 1 305 and 
view only those voxels within a desired section. The slicing area can be completely 
user-defined. 

Adding an opacity coefficient can also be used in other ways to aid in 
the exploration of a virtual system. If waste material is present and has a density as 
other properties within a certain known range, the waste can be made transparent to 
the virtual camera by changing its opacity coefficient during the examination. This 
will allow the patient to avoid ingesting a bowel cleansing agent before the procedure 
and make the examination faster and easier. Other objects can be similarly made to 
disappear depending upon the actual application. Additionally, some objects like 
polyps could be enhanced electronically by a contrast agent followed by a use of an 
appropriate transfer function. 

Figure 14 shows a system for perforating the virtual examination of an 
object such as a human organ using the techniques described in this specification. 
Patient 1401 lies down on a platform 1402 while scanning device 1405 scans the area 
that contains the organ or organs which are to be examined. The scanning device 
1405 contains a scanning portion 1403 which actually takes images of the patient and 
an electronics portion 1406. Electronics portion 1406 comprises an interface 1407, a 
central processing unit 1409. a memory 141 1 for temporarily storing the scanning ' 
data, and a second interface 1 413 for sending data to the virtual navigation platform. 
Interface 1407 and 1413 could be included in a single interface component or could be 
the same component The components in portion 1406 are connected together with 
conventional connectors. 

In system 1400, the data provided from the scanning portion of device 
1403 is transferred to portion 1405 for processing and is stored in memory 141 1. 
Central processing unit 1409 converts the scanned 2D data to 3D voxel data and stores 
the results in another portion of memory 141 1. Alternatively, the converted data 
could be directly sent to interface unit 1413 to be transferred to the virtual navigation 
terminal 1416. The conversion ofthe 2D data could also take place at the virtual 
navigation terminal 14 16 after being transmitted from interface 1413. In the preferred 
embodiment, the converted data is transmitted over carrier 1414 to the virtual 
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navigation terminal 1416 in order for an operator to perform the virtual examination. 
The data could also be transported in other conventional ways such as storing the data 
on a storage medium and physically transporting it to terminal 1416 or by using 
satellite transmissions. 

5 The scanned data may not be converted to its 3D representation until 

the visualization rendering engine requires it to be in 3D form. This saves 
computational steps and memory storage space. 

Virtual navigation terminal 1416 includes a screen for viewing the 
virtual organ or other scanned image, an electronics portion 1 41 5 and interface control 

10 1419 such as a keyboard, mouse or spaceball. Electronics portion 1415 comprises a 
interface port 1421, a central processing unit 1423, other components 1427 necessary 
to run the terminal and a memory 1425. The components in terminal 1416 are 
connected together with conventional connectors. The converted voxel data is 
received in interface port 1421 and stored in memory 1425. The central processor unit 

15 1 423 then assembles the 3D voxels into a virtual representation and runs the 
submarine camera model as described in Figures 2 and 3 to perform the virtual 
examination. As the submarine camera travels through the virtual organ, the visibility 
technique as described in Figure 9 is used to compute only those areas which are 
visible from the virtual camera and displays them on screen 1417. A graphics 

20 accelerator can also be used in generating the representations, The operator can use 
interface device 1419 to indicate which portion of the scanned body is desired to be 
explored. The interface device 1419 can further be used to control and move the 
submarine camera as desired as discussed in Figure 2 and its accompanying 
description. Terminal portion 1415 can be the Cube-4 dedicated system box, 

25 generally available from the Department of Computer Science at the State University 
of New York at Stony Brook. 

Scanning device 1405 and terminal 1416, or parts thereof, can be part 
of the same unit A single platform would be used to receive the scan image data, 
connect it to 3D voxels if necessary and perform the guided navigation. 

30 An important feature in system 1 400 is that the virtual organ can be 

examined at a later time without the presence of the patient. Additionally, the virtual 
examination could take place while the patient is being scanned. The scan data can 
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also be sent to multiple terminals which would allow more than one doctor to view the 
inside of the organ simultaneously. Thus a doctor in New York could be looking at 
the same portion of a patient's organ at the same time with a doctor in California while 
discussing the case. Alternatively, the data can be viewed at different times. Two or 
5 more doctors could perform their own examination of the same data in a difficult case. 
Multiple virtual navigation terminals could be used to view the same scan data. By 
reproducing the organ as a virtual organ with a discrete set of data, there are a 
multitude of benefits in areas such as accuracy, cost and possible data manipulations. 

The above described techniques can be further enhanced in virtual 
colonoscopy applications through the use of an unproved electronic colon cleansing 
technique which employs modified bowel preparation operations followed by image 
segmentation operations, such that fluid and stool remaining in the colon during a 
computed tomographic (CT) or magnetic resonance imaging (MTU) scan can be ' 
detected and removed from the vinual colonoscopy images. Through the use of such 
1 5 techniques, conventional physical washing of the colon, and its associated 
inconvenience and discomfort, is minimized or completely avoided. 

Referring to Figure 15, the first step in electronic colon cleansing is 
bowel preparation (step 1510), which takes place prior to conducting the CT or 
magnetic resonance imaging (MRI) scan and is intended to create a condition where 
residual stool and fluid remaining in the colon present significantly different image 
properties from that of the gas-filled colon interior and colon wall. An exemplary 
bowel preparation operation includes ingesting three 250 cc doses of Barium Sulfate 
suspension of 2. 1 % W/V, such as manufactured by E-Z-EM, Inc.,of Westbury, New 
York, during the day prior the CT or MRI scan. The three doses should be spread out 
over the course of the day and can be ingested along with three meals, respectively. 
The Barium Sulfate serves to enhance the images of any stool which remains in the 
colon. In addition to the intake of Barium Sulfate, fluid intake is preferably increased 
during the day prior to the CT or MRI scan. Cranberry juice is known to provide 
increased bowel fluids and is preferred, although water can also be ingested. In both 
the evening prior to the CT scan and the morning of the CT scan, 60 ml of a 
Diatrizoate Meglumine and Diaztrizoate Sodium Solution, which is commercially 
available as MD-Gastroview, manufactured by Mallinckrodt, Inc. of St. Louis, 
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Missouri, can be consumed to enhance image properties of the colonic fluid. Sodium 
phosphate can also be added to the solution to tiquilize the stool in the colon, which 
provides for more uniform enhancement of the colonic fluid and residual stool. 

The above described exemplary preliminary bowel preparation 
5 operation can obviate the need for conventional colonic washing protocols, which can 
call for the ingestion of a gallon of Golytely solution prior to a CT scan. 

Just prior to conducting the CT scan, an intravenous injection of 1 ml 
of Glucagon, manufactured by Ely Lily and Company, of Indianapolis, Indiana can be 
administered to minimize colon collapse. Then, the colon can be inflated using 

1 0 approximately 1 OOOcc of compressed gas, such as C0 2 , or room air, which can be 
introduced through a rectum tube. At this point, a conventional CT scan is perfoimed 
to acquire data from the region of the colon (step 1520). For example, data can be 
acquired using a GE/CTI spiral mode scanner operating in a helical mode of 5mm, 
1 .5-2.0: 1 pitch, where the pitch is adjusted based upon the patient's height in a known 

1 5 manner. A routine imaging protocol of 120 kVp and 200-280 ma can be utilized for 
this operation. The data can be acquired and reconstructed as 1mm thick slice images 
having an array size of 5 12x5 12 pixels in the field of view, which varies from 34 to 40 
cm depending on the patient's size, the number of such slices generally varies under 
these conditions from 300 to 450, depending on the patient's height. The image data 

20 set is converted to volume elements or voxels (step 1 530). 

Image segmentation can be performed in a number of ways. In one 
present method of image segmentation, a local neighbor technique is used to classify 
voxels of the image data in accordance with similar intensity values. In this method, 
each voxel of an acquired image is evaluated with respect to a group of neighbor 

25 voxels. The voxel of interest is referred to as the central voxel and has an associated 
intensity value. A classification indicator for each voxel is established by comparing 
the value of the central voxel to each of its neighbors. If the neighbor has the same 
value as the central voxel, the value of the classification indicator is incremented. 
However, if the neighbor has a different value from the central voxel, the 

30 classification indicator for the central voxel is decremented. The central voxel is then 
classified to that category which has the maximum indicator value, which indicates 
the most uniform neighborhood among the local neighbors. Each classification is 
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indicative of a particular intensity range, which in turn is representative of one or 
more material types being imaged The method can be further enhanced by 
employing a mixture probability function to the similarity classifications derived. 

An alternate process of image segmentation is performed as two major 
operations: low level processing and high level feature extraction. During low level 
processing, regions outside the body contour are eliminated from further processing 
and voxels within the body contour are roughly categorized in accordance with well 
defined classes of intensity characteristics. For example, a CT scan of the abdominal 
region generates a data set which tends to exhibit a well defined intensity distribution. 
The graph of Figure 16 illustrates such an intensity distribution as an exemplary 
histogram having four, well defined peaks, 1602, 1604, 1606, 1608, which can be 
classified according to intensity thresholds. 

The voxels of the abdominal CT data set are roughly classified as four 
clusters by intensity thresholds (step 1540). For example, Cluster 1 can include 
voxels whose intensities are below 140. This cluster generally corresponds to the ' 
lowest density regions within the interior of the gas filled colon. Cluster 2 can include 
voxels which have intensity values in excess of 2200. These intensity values 
correspond to the enhanced stool and fluid within the colon as well as bone. Cluster 
3 can include voxels with intensities in the range of about 900 to about 1080. This 
intensity range generally represents soft tissues, such as fat and muscle, which are 
unlikely to be associated with the colon. The remaining voxels can then be grouped 
together as cluster 4, which are likely to be associated with the colon wall (including 
mucosa and partial volume mixtures around the colon wall) as well as lung tissue and 
soft bones. 

Clusters 1 and 3 are not particularly valuable in identifying the colon 
wall and, therefore are not subject to substantial processing during image 
segmentation procedures for virtual colonoscopy. The voxels associated with cluster 
2 are important for segregating stool and fluid from the colon wall and are processed 
further during the high-level feature extraction operations. Low level processing is 
concentrated on the fourth cluster, which has the highest likelihood of corresponding 
to colon tissue (step 1550). 
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For each voxel in the fourth cluster, an intensity vector is generated 
using itself and its neighbors. The intensity vector provides an indication of the 
change in intensity in the neighborhood proximate a given voxel. The number of 
neighbor voxels which are used to establish the intensity vector is not critical, but 

5 involves a tradeoff between processing overhead and accuracy. For example, a simple 
voxel intensity vector can be established with seven (7) voxels, which includes the 
voxel of interest, its front and back neighbors, its left and right neighbors and its top 
and bottom neighbors, all surrounding the voxel of interest on three mutually 
perpendicular axes. Figure 17 is a perspective view illustrating an exemplary intensity 

1 0 vector in the form of a 25 voxel intensity vector model, which includes the selected 
voxel 1702 as well as its first, second and third order neighbors. The selected voxel 
1 702 is the central point of this model and is referred to as the fixed voxel. A planar 
slice of voxels, which includes 12 neighbors on the same plane as the fixed voxel, is 
referred to as the fixed slice 1 704. On adjacent planes to the fixed slice are two 

15 nearest slices 1706, having five voxels each. Adjacent to the first nearest slices 1706 
are two second nearest slices 1708, each having a single voxel. The collection of 
intensity vectors for each voxel in the fourth cluster is referred to as a local vector 
series. 

Because the data set for an abdominal image generally includes more 
20 than 300 slice images, each with a 5 1 2 x 5 12 voxel array, and each voxel having an 
associated 25 voxel local vector, it is desirable to perform feature analysis (step 1570) 
on the local vector series to reduce the computational burden. One such feature 
analysis is a principal component analysis (PCA), which can be applied to the local 
vector series to determine the dimension of a feature vector series and an orthogonal 
25 transformation matrix for the voxels of cluster 4. 

It has been found that the histogram (Figure 16) of the CT image 
intensities tends to be fairly constant from patient to patient for a particular scanner, 
given equivalent preparation and scanning parameters. Relying on this observation, 
an orthogonal transformation matrix can be established which is a predetermined 
30 matrix determined by using several sets of training data acquired using the same 
scanner under similar conditions. From this data, a transformation matrix, such as a 
Karlhunen-Loeve (K-L) transformation, can be generated in a known manner. The 
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transformation matrix is applied to the local vector series to generate feature vector 
series. Once in the feature-vector space domain, vector quantization techniques can 
be used to classify the feature vector series. 

An analytical, self-adaptive algorithm can be used for the classification 
5 of the feature vectors. In defining this algorithm, let {X f eR 4 :i = U.3.....N} be the 
series of the feature vectors, where N is the number of feature vectors; K. denotes the 
maximum number of classes; and T is a threshold which is adaptive to the data set 
For each class, a representative element is generated by the algorithm. Let a* be a 
representative element of class k and n k be the number of feature vectors in that class. 
The algorithm can then be outlined as: 
Set n^l; a^X^X*!; 

obtain the class number K and class parameters (a*, n,,) 
for(r=l;/<N;j++) 

for j<K; 

calculate d } = distlX^a^ ; 

end for 

index=aic min d,; 

update class parameters: 



10 



15 



. 1 

iMeX ' ~n 7T X ( "in*" • a i«,e X + X i > • 

index 

"index* "index 



end if 

else 

generate new class 

nz . , = 1 ; 
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end else 

end for 

3 . label each feature vector to a class according to the nearest neighbor rule 
for (i = l; i<W; i ++) 
5 for j<K; j + + ) 

calculate = dist [X if ; 

end for 

index- arc xmndj. 

label voxel i to class index. 

10 end for 

In this algorithm, dist(x,y) is the Euclidean distance between vector x 
and y and arc min d y gives the integer ; which realizes the minimum value of d,. 

The above described algorithm is dependent only on the parameters T and K. 
However, the value of K, which relates to the number of classes within each voxel 

1 5 cluster, is not critical and can be set to a constant value, such as K-l 8. However, T, 
which is the vector similarity threshold, greatly influences the classification results. If 
the selected value of T is too large, only a single class will be generated. On the other 
hand, if the value of T is too small, the resulting classes will exhibit undesirable 
redundancy. By setting the value of T to be equal to the maximum component 

20 variance of the feature vector series, the maximum number of distinct classes results. 

As a result of the initial classification process, each voxel within the 
selected cluster is assigned to a class (step 1 570). In the exemplary case of virtual 
colonoscopy, there are several classes within cluster 4. Thus, the next task is to 
determine which of the several classes in cluster 4 corresponds to the colon wall. The 

25 first coordinate of the feature vector, which is that coordinate of the feature vector 
exhibiting the highest variance, reflects the information of the average of the 3D local 
voxel intensities. The remaining coordinates of the feature vector contain the 
information of directional intensity change within the local neighbors. Because the 
colon wall voxels for the interior of the colon are generally in close proximity to the 
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gas voxels of cluster 1, a threshold interval can be determined by data samples 
selected from typical colon wall intensities of a typical CT data set to roughly 
distinguish colon wall voxel candidates. The particular threshold value is selected for 
each particular imaging protocol and device. This threshold interval can then applied 
5 to all CT data sets (acquired from the same machine, using the same imaging 
protocol). If the first coordinate of the representative element is located in the 
threshold interval, the corresponding class is regarded as the colon wall class and all 
voxels in that class are labeled as colon wall-like voxels. 

Each colon wall-like voxel is a candidate to be a colon wall voxel. 

10 There are three possible outcomes of not belonging to the colon wall. The first case 
relates to voxels which are close to the stool/liquid inside the colon. The second case 
occurs when voxels are in the lung tissue regions. The third case represents mucosa 
voxels. Clearly then, low level classification carries a degree of classification 
uncertainty. The causes of the low-level classification uncertainty vary. For example, 

1 5 a partial-volume effect resulting from voxels containing more than one material type 
(i.e., fluid and colon wall) leads to the first case of uncertainty. The second and the 
third cases of uncertainty are due to both the partial volume effect as well as the low 
contrast of CT images. To resolve the uncertainty, additional information is needed. 
Thus, a high-level feature extraction procedure is used in the present method to further 

20 distinguish candidates for the colon wall from other colon wall-like voxels, based on a 
priori anatomical knowledge of the CT images (step 1580). 

An initial step of the high-level feature extraction procedure can be to 
eliminate the region of lung tissue from the low-level classification results. Figure 
1 8A is an exemplary slice image clearly illustrating the lung region 1802. The lung 

25 region 1802 is identifiable as a generally contiguous three dimensional volume 
enclosed by colon wall-like voxels, as illustrated in Figure 18B. Given this 
characteristic, the lung region can be identified using a region growing strategy. The 
first step in this technique is to find a seed voxel within the region of growing. 
Preferably, the operator performing the CT imaging scan sets the imaging range such 

30 that the top most slice of the CT scan does not contain any colon voxels. As the 
interior of lung should be filled with air, the seed is provided by the low-level 
classification simply by selecting an air voxel. Once the lung region outline of Figure 
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1 8B is determined, the lung volume can be removed from the image slice (Figure 
18C). 

A next step in performing high-level feature extraction can be to 
separate the bone voxels from enhanced stool/fluid voxels in cluster 2. The bone 

5 tissue voxels 1902 are generally relatively far away from the colon wall and resides 
outside the colon volume. To the contrary, the residual stool 1 906 and fluid 1 904 are 
enclosed inside the colon volume. Combining the a priori proximity information and 
the colon wall information obtained from the low-level classification process, a rough 
colon wall volume is generated. Any voxel separated by more than a predetermined 

1 0 number (e.g., 3) of voxel units from the colon wall, and outside the colon volume, 
will be labeled as bone and then removed from the image. The remaining voxels in 
cluster 2 can be assumed to represent stool and fluid within the colon volume (see 
Figures 19A-C). 

The voxels within the colon volume identified as stool 1906 and fluid 

15 1 904 can be removed from the image to generate a clean colon lumen and colon wall 
image. In general, there are two kinds of stool/fluid regions. One region type is small 
residual areas of stool 1 906 attached to the colon wall. The other region type is large 
volumes of fluid 1904, which collect in basin-like colonic folds (see Figures 19A-C). 

The attached residual stool regions 1906 can be identified and removed 

20 because they are inside the rough colon volume generated during the low-level 
classification process. The fluid 1906 in the basin-like colon fold usually has a 
horizontal surface 1908 due to the effect of gravity. Above the surface is always a gas 
region, which exhibits a very high contrast to the fluid intensity. Thus, the surface 
interface of the fluid regions can be easily marked. 

25 Using a region growing strategy, the contour of the attached stool 

regions 1906 can be outlined, and the part which is away from the colon wall volume 
can be removed. Similarly, the contour of the fluid regions 1904 can also be outlined. 
After eliminating the horizontal surfaces 1908, the colon wall contour is revealed and 
the clean colon wall is obtained. 

30 It is difficult to distinguish the mucosa voxels from the colon wall 

voxels. Even though the above three dimensional processing can remove some 
mucosa voxels, it is difficult to remove all mucosa voxels. In optical colonoscopy, 
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physicians dirccUy inspect the colonic mucosa and search for lesions based on the 
color and texture of the mucosa In virtual colonoscopy, most mucosa voxels on the 
colon wall can be left intact in order to preserve more information. This can be very 
useful for three dimensional volume rendering. 
5 From the segmented colon wall volume, the inner surface, the outer 

surface and the wall itself of the colon can be extracted and viewed as a virtual object. 
This provides a distinct advantage over conventional optical colonoscopy in that the 
exterior wall of the colon can be examined as well as the interior wall. Furthermore, 
the colon wall and the colon lumen can be obtained separately from the segmentation. 

1 0 Because the colon is substantially evacuated prior to imaging, a 

commonly encountered problem is that the colon lumen collapses in spots. While the 
inflation of the colon with compressed gas, such as air or CO,, reduces the frequency 
of collapsed regions, such areas still occur. In performing a virtual colonoscopy, it is 
desirable to automatically maintain a flight path through the collapsed regions and it is 

15 also desirable to use the scanned image data to at least partially recreate the colon 
lumen in the collapsed regions. Since the above described image segmentation 
methods effectively derive both the interior and exterior of the colon wall, this 
information can be used to enhance the generation of the fly path through the 
collapsed regions. 

20 In tending the flight path through collapsed regions of the colon or 

expanding a collapsed region of the colon, the first step is to detect a collapsed region. 
Using the premise that the grayscale values of the image data from around the outside 
of the colon wall change much more dramatically than the grayscale values within the 
colon wall itself, as well as in other regions such as fat, muscle and other kinds of 

25 tissue, an entropy analysis can be used to detect areas of colon collapse. 

The degree of change in grayscale value, for example along the 
centerline, can be expressed and measured by an entropy value. To calculate an 
entropy value, voxels on the outer surface of the colon wall are selected. Such points 
are identified from the above described image segmentation techniques. A 5x5x5 

30 cubic window can be applied to the pixels, centered on the pixel of interest. Prior to 
calculating the entropy value, a smaller (3x3x3) window can be applied to the pixels 
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of interest in order to filter out noise from the image data The entropy value of a 
selected window about the pixel can then be determined by the equation: 

r=Ec(i)ln(C(i)) 

i 



where E is the entropy and C(i) is the number of points in the window with the 
grayscale of i (i=0,l,2,. . ., 255). The calculated entropy values for each window are 
5 then compared against a predetermined threshold value. For regions of air, the entropy 
values will be fairly low, when compared to regions of tissue. Therefore, along the 
centerline of the colon lumen, when the entropy values increase and exceed the 
predetermined threshold value, a collapsed region is indicated. The exact value of the 
threshold is not critical and will depend in part on the imaging protocol and particulars 

10 of the imaging device. 

Once a collapsed region is detected, the previously determined 
centerline flight path can be extended through the region by piercing through the 
center of the collapse with a one voxel wide navigation line. 

In addition to automatically continuing the flight path of the virtual 

1 5 camera through the colon lumen, the region of colon collapse can be virtually opened 
using a physical modeling technique to recover some of the properties of the collapsed 
region. In this technique, a model of the physical properties of the colon wall is 
developed. From this model, parameters of motion, mass density, damping density, 
stretching and bending coefficients are estimated for a Lagrange equation. Then, an 

20 expanding force model (i.e., gas or fluid, such as air, pumped into the colon) is 
formulated and applied in accordance with the elastic properties of the colon, as 
defined by the Lagrange equation, such that the collapsed region of the colon image is 
restored to its natural shape. 

To model the colon, a finite-element model can be applied to the 

25 collapsed or obstructed regions of the colon lumen. This can be performed by 

sampling the elements in a regular grid, such as an 8 voxel brick, and then applying 
traditional volume rendering techniques. Alternatively, an irregular volume 
representation approach, such as tetrahedrons can be applied to the collapsed regions. 
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In applying the external force (air pumping) model to the colon model, 
the magnitude of the external force is first determined to properly separate the 
collapsed colon wall regions. A three dimensional growing model can be used to 
trace the internal and external colon wall surfaces in a parallel manner. The respective 
5 surfaces are marked from a starting point at the collapsed region to a growing source 
point, and the force model is applied to expand the surfaces in a like and natural 
manner. The region between the internal and external surfaces, i.e., the colon wall, 
are classified as sharing regions. The external repulsive force model is applied to 
these sharing regions to separate and expand the collapsed colon wall segments in a 
10 natural manner. 

To more clearly visualize the features of a virtual object, such as the 
colon, which is subjected to virtual examination, it is advantageous to provide a 
rendering of the various textures of the object. Such textures, which can be observed 
in the color images presented during optical colonoscopy, are often lost in the black 
15 and white, grey scale images provided by the CT image data. Thus a system and 
method for texture imaging during virtual examination is required. 

Figure 20 is a flow chart depicting a present method for generating 
virtual objects having a texture component. The purpose of this method is to map 
textures obtained by optical colonoscopy images in the red-green-blue (RGB) color 
20 space, as for example from the Visible Human, onto the gray scale monochrome CT 
image data used to generate virtual objects. The optical colonoscopy images are 
acquired by conventional digital image acquistion techniques, such as by a digital 
"frame grabber" 1429 which receives analog optical images from a camera, such as a 
video camera, and converts the image to digital data which can be provided to CPU 
25 1423 via interface port 1431 (Figure 14). The first step in this process is to segment 
the CT image data (step 201 0). The above described image segmentation techniques 
can be applied to choose intensity thresholds in the grey scale image to classify the CT 
image data into various tissue types, such as bone, colon wall tissue, air, and the like. 

In addition to performing image segmentation on the CT image data, 
the texture fearures of the optical image need to be extracted from the optical image 
data (step 2020). To do this, a gausian filter can be applied to the optical image data 
followed by subsampling to decompose the data into a multiresolutional pyramid. A 



30 
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laplacian filter and steerable filter can also be applied to the multiresolutional pyramid 
to obtain oriented and non-oriented features of the data. While this method is 
effective at extracting and capturing the texture features, the implementation of this 
approach requires a large amount of memory and processing power. 
5 An alternative approach to extracting the texture features from the 

optical image is to utilize a wavelet transform. However, while wavelet 
transformations are generally computationally efficient, conventional wavelet 
transforms are limited in that they only capture features with orientations parallel to 
the axes and cannot be applied directly to a region of interest. To overcome these 

1 0 limitations, a non-separable filter can be employed. For example, a lifting shcme cam 
be employed to build filter banks for wavelets transform in any dimension using a two 
step, prediction and updating approach. Such filter banks can be synthesized by the 
Boor-Rom algorithm for multidimensional polynomial interpolation. 

After the textural features are extracted from the optical image data, 

1 5 models must be generated to describe these features (step 2030). This can be 

performed, for example, by using a non-parametric multi-scale statistical model which 
is based on estimating and manipulating the entropy of non-Gaussian distributions 
attributable to the natural textures. 

Once texture models are generated from the optical image data, texture 

20 matching must be performed to correlate these models to the segmented CT image 
data (step 2050). In regions of the CT image data where the texture is continuous, 
corresponding classes of texture are easily matched. However, in boundary regions 
between two or more texture regions, the process is more complex. Segmentation of 
the CT data around a boundary region often leads to data which is fuzzy, i.e., the 

25 results reflect a percentage of texture from each material or tissue and vary depending 
on the various weighting of each. The weighting percentage can be used to set the 
importance of matching criteria. 

In the case of the non-parametric multi-scale statistical model, the cross 
entropy or a Kullback-Leiber divergance algorithm can be used to measure the 

30 distribution of different textures in a boundary region. 
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After texture matching, texture synthesis is performed on the CT image 
data (step 2050). This is done by fusing the textures from the optical image data in to 
the CT image data. For isotropic texture patterns, such as presented by bone, the 
texture can be sampled directly from the optical data to the segmented CT image data. 
5 For unisotropic texture regions, such as colon mucosa, a multiresolution sampling 
procedure is preferred. In this process, selective resampling for homogenous and 
heterogenous regions is employed. 

In addition to enhanced imaging, the above described techniques can 
also form the basis of a system for performing virtual electronic biopsy of a region 

10 being examined to effect a flexible and non-invasive biopsy. Volume rendering 
techniques employ a defined transfer function to map different ranges of sample 
values of the original volume data to different colors and opacities. Once a suspicious 
area is detected during virtual examination, the physician can interactively change'the 
transfer function used during the volume rendering procedure such that the wall being 

1 5 viewed becomes substantially transparent and the interior of the area can be viewed. 

In addition to performing virtual biopsy, the present system and 
methods can be extended to perform automated polyp detection. Polyps which occur, 
for example, within the colon, are generally small convex hill-like structures 
extending from the colon wall. This geometry is distinct from the fold of the colon 

20 wall. Thus, a differential geometry model can be used to detect such polyps on the 
colon wall. 

The surface of the colon lumen can be represented using a C-2 
smoothness surface model. In this model, each voxel on the surface has an associated 
geometrical feature which has a Gaussian curvature, referred to as Gaussian curvature 
25 fields. A convex hill on the surface, which may be indicative of a polyp, possesses a 
unique local feature in the Gaussian curvature fields. Accordingly, by searching the 
Gausian curvature fields for specific local features, polyps can be detected. 

Each of the foregoing methods can be implemented using a system as 
illustrated in Figure 14, with appropriate software being provided to control the 
30 operation of CPU 1 409 and CPU 1 423. 
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The foregoing merely illustrates the principles of the invention. It will 
thus be appreciated that those skilled in the ait will be able to devise numerous 
systems, apparatus and methods which, although not explicitly shown or described 
herein, embody the principles of the invention and are thus within the spirit and scope 
5 of the invention as defined by its claims. 

For example, the methods and systems described herein could be 
applied to virtually examine an animal, fish or inanimate object. Besides the stated 
uses in the medical field, applications of the technique could be used to detect the 
contents of sealed objects which cannot be opened. The technique could also be used 
10 inside an architectural structure such as a building or cavern and enable the operator to 
navigate through the structure. 
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CLAIMS 



1 . A method for electronically cleansing a virtual object, formed from image 
data, comprising: 

converting the image data to a plurality of volume elements forming the virtual 
5 object, each volume element having an intensity value; 

classifying the volume elements into a plurality of clusters in accordance with 
the intensity values; and 

removing at least one cluster of volume elements from the virtual object. 



10 



2. The method for electronically cleansing a virtual object according to claim 1 , 
wherein the classifying operation further comprises evaluating a plurality of volume 
elements of the image data with respect to a plurality of neighboring volume elements 
to determine a neighborhood similarity value for the volume elements. 



3. The method for electronically cleansing a virtual object according to claim 2, 
wherein the clusters are classified according to the similarity value of the volume 

15 elements. 

4. The method for electronically cleansing a virtual object according to claim 2, 
wherein the clusters are further classified by applying a mixture probability function to 
the clusters to classify voxels whose intensity value results from inclusion of more 
than one material type. 

20 5. The method for electronically cleansing a virtual object according to claim 1 , 
wherein the classifying operation further comprises: 

performing feature vector analysis on at least one of the clusters which 
include image data for a material of interest; and 

performing high level feature extraction to remove volume elements from the 
25 virtual object which are not substantially indicative of the material of interest. 
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6. The method for electronically cleansing a virtual object according to claim 5, 
wherein the image data represents a region of the human body including at least a 
portion of the colon and the material of interest is tissue of a colon. 



7. The method for electronically cleansing a virtual object according to claim 1 , 
5 wherein the image data represents a region of the human body including at least a 

portion of the colon. 

8. The method for electronically cleansing a virtual object according to claim 7, 
wherein the removing operation removes volume elements representing at least one of 
intracolonic fluid, residual stool within the colon, bone, and non-colonic tissue. 

10 9. A method for preparing image data for virtual colonoscopy comprising: 
acquiring an image data set including at least a portion of the colon; 
converting the image data set to a plurality of volume elements, each volume element 
having an intensity value; 

classifying the volume elements into a plurality of clusters in accordance with 
1 5 the intensity values, each cluster representing at least one material proximate the 
colon; and 

removing at least one cluster of volume elements from the image data set. 

10. The method for preparing image data for virtual colonoscopy, according to 
claim 9, further comprising enhancing the intensity values of volume elements of fluid 

20 and stool residing within the colon prior to the acquiring operation. 

1 1 . The method of performing virtual colonoscopy, according to claim 1 0, wherein 
the enhancing operation comprises the step of a patient ingesting material to enhance 
the image intensity of stool and fluid within the colon. 

12. The method of performing virtual colonoscopy, according to claim 1 1, wherein 
the ingesting step includes ingesting at least one of Barium Sulfate, Diatrizoate 
Meglumine and Diaztrizoate Sodium solution. 



25 
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13. The method of perfoiming virtual colonoscopy according to claim 10, wherein 
the at least one material proximate the colon includes colon wall tissue and at least 
one of bone, fluid, stool and non-colonic material. 



14. The method of performing virtual colonoscopy according to claim 13, wherein 
one of the plurality of clusters includes the intensity enhanced volume elements 
representing fluid and stool, and this cluster is removed during the removing 
operation. 



1 5. The method of perfoiming virtual colonoscopy according to claim 9, wherein 
one of the plurality of clusters includes volume elements of the colon wall and other 

1 0 colon wall-like materials and a further classification operation is performed on this 
cluster to identify the volume elements of the colon wall. 

16. The method of performing virtual colonoscopy according to claim 1 5, further 
comprising: 

identifying the interior of the colon; 

15 generating a centerline for navigating through the interior of the colon; 

detecting a collapsed region of the interior of the colon; and 

extending the centerline through the collapsed region. 

17. The method of performing virtual colonoscopy according to claim 1 6, wherein 

entropy values are calculated for the intensity values proximate the centerline and the 

20 detecting operation includes identifying at least one of the entropy values being above 
a threshold value. 



18. The method of performing virtual colonoscopy according to claim 17, further 
comprising virtually expanding a detected collapsed region of the colon in accordance 
with properties of the colon wall. 
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1 9. A method of mapping optical texture properties from at least one optical image 
to 

an acquired monochrome data set comprising: 

segmenting the acquired monochrome data set into a plurality of classifications 
5 representing a plurality of textures; 

segmenting the optical image into a plurality of color classifications 
representing a second plurality of textures; 

generating a texture model for the plurality of color classifications; 
matching the texture models to the plurality of classifications of the 
10 monochrome image data; and 

applying the texture models to the monochrome image data. 



20. An imaging system including an image segmentation feature, comprising: 
an imaging scanner for acquiring image data; 

a processor, said processor converting the image data into a plurality of 
1 5 volume elements forming a volume element data set, each volume element having an 
intensity value, the processor performing an image segmentation operation on the 
volume element data set comprising classifying the volume elements into a plurality 
of clusters in accordance with the intensity values and removing at least one cluster of 
volume elements from the image data; and 
20 a display unit operatively coupled to the processor for displaying a 

representation 

of the image data with the at least one cluster of volume elements removed. 

21. The imaging system according to claim 20, wherein the classifying operation 
performed by the processor further comprises evaluating a plurality of volume 

25 elements of the image data with respect to a plurality of neighboring volume elements 
to determine a neighborhood similarity value for the volume element. 



22. The imaging system according to claim 2 1 , wherein the classifying operation 
performed by the processor classifies the clusters according to the similarity value of 
the volume elements. 
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23. The imaging system according to claim 20, wherein the classifying operation 
performed by the processor includes a mixture probability algorithm to classify voxels 
whose intensity value results from inclusion of more than one material type. 

24. The imaging system according to claim 20, wherein the classifying operation 
5 performed by the processor further comprises: 

performing feature vector analysis on at least one of the clusters which 
include image data for a material of interest; and 

performing high level feature extraction to remove volume elements from the 
image which are not substantially indicative of the material of interest. 

10 25. The imaging system according to claim 24, wherein the imaging scanner is 
adapted to acquire image data of a human body including at least a portion of the 
colon and the material of interest is tissue of a colon. 



26. The imaging system according to claim 20, wherein the imaging scanner is 
adapted to acquire image data of a region of the human body including at least a 
1 5 portion of the colon. 



27. The imaging system according to claim 20, wherein the classifying operation 
performed by the processor removes volume elements from the volume element data 
set which represent at least one of intracolonic fluid, residual stool within the colon, 
bone, and non-colonic tissue. 



25 



lone 



20 28. An imaging system for mapping optical texture properties from at least < 
optical image to an acquired monochrome data set comprising: 
an imaging scanner for acquiring the monochrome data set; 
a processor, said processor segmenting the acquired monochrome data set into 
a plurality of classifications representing a plurality of textures, segmenting the optical 
image into a plurality of color classifications representing a second plurality of 
textures, generating a texture model for the plurality of color classifications, matching 
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the texture models to the plurality of classifications of the monochrome image data, 
and applying the texture models to the monochrome image data; and 

a display unit operatively coupled to the processor for displaying a 
representation 
5 of the image data with the texture models applied. 

29. The imaging system for mapping optical texture properties according to claim 
28, wherein the imaging scanner is a computed tomography scanner. 

30. The imaging system for mapping optical texture properties according to claim 
28, wherein the imaging scanner is a magnetic resonance imaging scanner. 

10 31. The imaging system for mapping optical texture properties according to claim 
28, wherein the image data with the texture models applied is a color representation of 
the object being imaged, 

32. A system for performing virtual colonoscopy comprising: 
an imaging scanner for acquiring image data of a colon; 

1 5 a processor, said processor receiving the image data and identifying the 

interior of the colon, generating a centerline for navigating through the interior of the 
colon, detecting a collapsed region of the interior of the colon, and extending the 
centerline through the collapsed region; 

a display unit operatively coupled to the processor for displaying a 

20 representation 
of the colon. 

33. The system for performing virtual colonoscopy according to claim 32, wherein 
the processor detects the collapsed region by calculating entropy values for the 
intensity values of the image data proximate the centerline and identifying at least one 

25 of the entropy values being above a threshold value. 
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34. The system for performing virtual colonoscopy according to claim 32, wherein 
the processor virtually expands a detected collapsed region of the colon in accordance 
with properties of the colon wall. 
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